What Are We Doing When We Assess Programming?

Considerable research has been devoted in recent decades to identifying optimal pedagogical strategies for teaching computer programming. Often the result of these efforts has been to conclude that we have made little progress, as our students consistently perform poorly on the assessments we apply to measure teaching efficacy. In this paper, we suggest that a significant contributor to this poor performance may be the methods of assessment, which do not reflect the knowledge and skills that a real programmer needs to write real code. We propose an alternative assessment format using Activity Diagrams that better reflects true programming ability. Our preliminary results indicate that these assessments correlate well with the ability to produce working code, while more traditional question formats do not.